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SPEAKER ADAPTATION USING WEIGHTED FEEDBACK 
Background 

Technical Field of the Invention: The present invention relates to speech 
recognition systems and, more particularly, to speaker adaptation using feedback. 

Background Art: Speech recognition systems using only Speaker Independent 
(SI) models are very sensitive to different speakers due to speaker characteristic 
variations. SI models typically use a Hidden Markov Model (HMM). Speaker 
adaptation is a process to adapt a SI model to a speaker dependent (SD) model to 
capture the physical characteristics of a given speaker. Speaker adaptation techniques 
can be used in supervised and unsupervised mode. In supervised mode, the correct 
transcription is known, while in unsupervised mode, no correct transcription is 
available. 

For reliable and robust speaker adaptation, large amounts of adaptation data are 
often required in order to cover the linguistic units of a given language. However for 
most practical applications, only a limited amount of adaptation data is available. 
Efficient use of the adaptation data becomes extremely important. The traditional 
adaptation schemes treat all the adaptation data indiscriminately, which results in some 
parts of the adaptation data being relatively under-trained or under-weighted. Usually 
the under represented words are more unlikely to be recognized by the decoder. 

The traditional adaptation scheme is as follows: 

1. Given some adaptation enrollment data and a SI model, collect statistics 
on the enrollment data and perform speaker adaptation on the SI model. 

2. Decoding the test utterances with the adapted acoustic model. 
Such a scheme uses the enrollment data only once and does not incorporate any 
feedback frorti decoding. It is fast in practice, but does not always provide good 
performance. 

Approaches to speaker adaptation include those described in J. L. Gauvain et al. 
"Maximum a posteriori estimation for multivariate Gaussian mixture observations of 
Markov Chain," IEEE Trans. On Speech and Audio Processing, Vol. 2, pp. 291-298; 
L.R. Bahl, et al., "A New Algorithm for the estimation of Hidden Markov Model 



Parameters," IEEE International Conference on Acoustics, Speech, and Signal 
Processing, pp. 493-496, 1988; and C.L. Leggetter et al., "Maximum likelihood linear 
regression for speaker adaptation of continuous density HMMs," Computer Speech and 
Language, Vol.9, pp. 171-185, 1995. In some of these approaches, errors included in 
recognizing a particular speaker's utterances are not considered. In a "corrective 
training" approach, such as in the above-recited L. R. Bahl et al. article, an error in 
recognition of the utterance may be considered, but a very complicated technique is 
used to compensate for it. Background articles on expectation maximization (EM) 
maximum likelihood (ML) are provided in the articles A.P. Dempster, et al., 
"Maximum likelihood from incomplete data via the EM algorithm," Journal of the 
Royal statistical Society, Series B 39, pp. 1-38, 1977; and N. Laird, 'The EM 
algorithm," Handbook of Statistics, vol. 9. Elsevier Science Publishers B.V. 1993. 

An iterative technique in speech recognition is to recognize utterances based on • 
an SI model and to create an SD model therefrom and then to apply the SD model to 
recognizing the utterances to create a more refined SD model and so forth. 

There is a need for improved techniques for speaker adaptation. Such improved 
techniques are described in this disclosure. 

Brief Description of the Dravt'ings 

The invention will be understood more fully from the detailed description given 
below and from the accompanying drawings of embodiments of the invention which, 
however, should not be taken to limit the invention to the specific embodiments 
described, but are for explanation and understanding only. 

FIG. 1 is a partial flow and partial block diagram representation of some 
embodiments of the invention. 

FIG. 2 illustrates a segment (e.g., a phone) of the utterances which includes 
multiple frames. 

FIG. 3 illustrates a section (e.g., word) of the utterances which includes multiple 
segments (e.g., phones). 

FIG. 4 is partial flow and partial block diagram representation similar to a 
portion of FIG. 1, but may allow multiple feedback passes. 



FIG. 5 is a high level schematic block diagram representation of a computer 
system that may be used in connection with some embodiments of the invention. 

FIG. 6 is a high level schematic representation of a hand-held computer system 
that may be used in connection with some embodiments of the invention. 

Detailed Description 

The present invention involves speaker adaptation whereby characteristics of an 
SI model can be adapted through consideration of adaptation enrollment data from a 
particular speaker to create an SD model. More particularly, the adaptation enrollment 
data is weighted according to errors detected in the recognized utterances. For those 
words (or utterances in the enrollment data set) that are not well learnt by speaker 
adaptation, as indicated by misrecognizing those words, the invention provides a way to 
incorporate the decoding feedback so that these words can be better adapted. When 
only limited amounts of enrollment data are available, this scheme of iterative 
bootstrapping makes better use of that limited data. The scheme can be extended to the 
unsupervised adaptation where references may contain errors. In some embodiments, 
an iterative adaptation scheme dynamically adjusts enrollment data to incorporate 
feedback from decoding on the enrollment data. 

In the following disclosure, the terra "some embodiments" or "other 
embodiments" means that a particular feature, structure, or characteristic described in 
connection with the embodiments is included in at least some embodiments, but not 
necessarily all embodiments, of the invention. The various appearances "some 
embodiments" are not necessarily all referring to the same embodiments. 
In the following disclosure, when the term phone is used, it could include all phonemes 
in a particular language or less than all the phonemes. To reduce complexity, some 
speech recognition systems do not recognize every phoneme in a particular language. 

The following four parts are used in some embodiments of the invention. A 
fifth part is used in still other embodiments. 

1 . Denote M as the initial SI (speaker independent) model and A as the 
enrollment data set. 

2. Perform speech recognition on data set A based on model M. 



3. Adjust A to A' according to the decoding results from part 2. 
Emphasizing or de-emphasizing certain parts of A with weights based on these results. 
The emphasizing/de-emphasizing is achieved by assigning a weight to each word in the 
adaptation data. How to calculate the weight will be discussed below. 

4. Adapt model M to M' using enrollment data A'. 

5. (optional) Repeat parts 3 and 4 with the updated M'. 

For example, FIG. 1 is represents some embodiments of the invention in a 
diagram which is partially a flow diagram and partially a block diagram. A dashed line 
represents a dividing line between acts occurring during an adaptation mode and a 
recognition (decoding) mode. The recognition phase occurs after the SD model is 
created in the adaptation phase. Note that microphone 14, processing block 18, and 
recognition block 22 are shown above and below the dashed line and may represent the 
same blocks at different times (before and after the conclusion of adaptation). In this 
disclosure, a block may be hardware or a combination of hardware and software. 

Referring to FIG. 1 above the dashed line, a speaker input such as microphone 
14 receives utterances of a particular speaker. The utterances are converted to digital 
signals U and may be otherwise processed according to well know techniques by 
processing block 18. Note that microphone 14 may be adjacent to the computer system 
that performs the acts illustrated in FIG. 1 or microphone may be remote from it. For 
example, microphone 14 may be in a telephone or remote other system. Processing 
block 18 provides the processed utterances U to a recognition block 22 and a weighting 
block 30. Utterances U may be stored in a wave file as a collection of utterances. Of 
course, there may be spaces of silence or lack of speech between the sections of the 
utterances. 

Recognition block 22 produces a recognized (hypothesized) phone string H 
based on the utterances U and an SI model. In a comparison and weight calculating 
block 26, recognized phone string H is compared with a reference (true) phone string R. 
The reference phone string is what the speaker is requested to read. ' A word-phone 
dictionary may be used to convert the reference word string into phones. Of course, 
there may be silences or lack of speech in the recognized and reference phone strings. 



Differences between the recognized and reference phone string can be 
determined in a variety of ways. In some embodiments, speech features in the 
recognized and reference phone strings are compared on a frame by frame level. 
Merely as an example, the frames may be on the order of about 25 milliseconds (ms), 
although various other frame durations could be used. A phone may be around 200 
milliseconds, although various other phone durations are possible. Accordingly, in 
some embodiments, there may be on the order of 10 frames per phone. A force 
alignment algorithm may be used to mark the time or place each phone (or word) 
happens in the utterances. The frames may contain a Gaussian feature vector. 

For example, referring to FIG. 2, portions of the reference string and recognized 
string for a series of frames are illustrated. The frames are arbitrarily labeled FI, F2, 
F3, . . ., FX-2, FX-1, and FX, wherein there may be several frames between frames F3 
and FX-2. The portions in a frame may be a feature extraction. Each portion (e.g., 
feature extraction) has characteristics (e.g., Gaussian), which are labeled "C." The 
particular number after the "C" is arbitrarily chosen. For example, in frame Fl, both 
the reference and recognized strings have characteristics C4. Accordingly, the 
comparison indicates that the characteristics of the reference and recognized strings are 
the same (S) for frame Fl . In frame F2, the characteristic of the reference string is C15 
and the characteristic of the recognized string is CU. Accordingly, the comparison 
indicates that the characteristics of the reference and recognized strings are the different 
(D) for frame F2. (Merely as an example, S could be "0" and D could be " 1 ," or 
various other schemes could be used.) Likewise, in frames F3, FX-2, and FX-1, the 
characteristics are the same and in frame FX, the characteristics are different. 

In some embodiments, a certain number of frames forms a segment. The 
segment may be a phone or other portion of the utterance. Referring to FIG. 2, as an 
example, a segment 1 may be formed of frames Fl, F2, F3, . . ., FX-2, FX-1, FX. As 
illustrated in FIG. 3, a section of the utterances may be formed of multiple segments. In 
some embodiments, the section is a word, although the invention is not so limited. 
Segments of silence or lack of speech can be used to indicate the boundary of a word. 



If a word includes a phone having an error (the characteristics of a frame of the 
reference and hypothesis in the word are different (see FIG. 2)), then the word is 
considered an error word, and the weight of the word is calculated. 

The weights assigned to the sections of utterances U may be calculated in block 
5 26 through various techniques. The following are some examples, although the 

invention is not limited to the examples. 

In some embodiments, the weight value for each word is estimated from the 
likelihood information of the references (the true input word string) and hypotheses (the 
word string decoded by the recognizer, may contain errors). 
10 1. Run a force alignment program on the reference stream to get statistics 

M= of the references. 

5 2. Decode the utterance to get statistics of the 1-best hypothesis. 

3. Align the 1-best hypothesis with the reference sentence to obtain the 
error words. 

5 4. Calculate the average likelihood difference per frame according to the 

equation (1) as follows: 

r„=_^£ ^ 



where ^£ is the log likelihood of hypothesis word n, is the beginning frame 

index (in time), and is the end frame index. Rl , and Re are the reference 
counter parts. Of course, the invention is not limited to the details of equation (1). 
Note that equation (1) involves hkelihoods, which are not necessarily probabilities. 
Equation (1) could be modified to involve probabilities. 

Next, the weight value Wj for misrecognized words of a particular speaker "i" is 
obtained by averaging over all the misrecognized words (error words) according to 
equation (2) as follows: 

W. = 1. * I|L«| (2), 
m n = \ 

wherein m may be the number of misrecognized words. Of course, the invention is not 
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limited to the details of equation (2). In equation (2), the sections are for words (e.g., 
"W" refers to words), but the sections could be something else. (See FIG. 3.) 

Note that in embodiments using equation (2), each misrecognized word may 
have the same averaged weight. Alternatively, different words could have different 
weights through applying the result of equation (1) more directly. 

Once the weights are calculated, the weights and places of error are provided to 
block 30. The reference string (or at least the portions of the reference string 
corresponding to the errors in the recognized stream) is communicated to block 30. In 
block 30, the utterance U is marked with the errors and corresponding weights are 
noted. The adaptation enrollment data (E) includes the marked utterances with 
corresponding weights (wU) and at least those portions of the reference stream (R*) 
that correspond to the errors in the utterance. The SI model and SD model may be 
Gaussian mixtures. The wavefile U may be transformed (e.g., through an FFT) from 
the time domain to the frequency domain. The weight w may be expressed as a floating 
point number. 

In adaptation box 34, the adaptation enrollment data is applied with the SI 
model to create the SD model according to known techniques, except that the 
enrollment data may have additional weights. In some embodiments, in the adaptation 
box 34, the error words are added w times to the SI model. In some embodiments, 
these weights are added to those of the SI model, although the invention is not limited 
to this. More complicated schemes could be used, but are not required. 

It is important to not give too much weight to the enrollment data, because they 
are based on limited sampling. 

In the above described embodiments, weights are only calculated for words for 
which there is an error in recognition. Alternatively, there could be weights (e.g., 
negative weights) for correctly recognized words. Note that in different embodiments 
the weights can be positive or negative depending on the scheme chosen. 

Once the SD model is calculated in an adaptation mode, it is applied on path 40 
for use by block 22 in a recognition mode, below the dashed line. 

FIG. 4 illustrates that the feedback can be performed more than once until 
differences between H and R are less than a threshold (see decision block 36). (It could 



be less than or equal to a threshold.) To determine whether the differences between H 
and R are less than a threshold, the various errors can be summed and then compared to 
a single threshold or different errors can be compared to separate thresholds. Other 
approaches could be used. 

Note that in FIG. 4, the input to recognition block 22 changes with each pass. 
The utterances may be stored for re-use. The inputs to the adaptation block 24 also 
changes, however, if the difference between H and R is less than a threshold, then the 
previous enrollment data is the one applied to path 40 for use during recognition mode 
(decoding). 

There are a variety of computer systems that may be used in training and using a 
speech recognition system. Merely as an example, FIG. 5 illustrates a highly schematic 
representation of a computer system 100 which includes a processor 1 14, memory 116, 
and input/output and control block 1 18. There may be a substantially amount of 
memory in processor 1 14 and memory 1 16 may represent both memory that is off the 
chip of processor 1 14 or memory that is partially on and partially off the chip of 
processor 1 14. (Or memory 116 could be completely on the chip of processor 1 14). At 
least some of the input/output and control block 118 could be on the same chip as 
processor 1 14, or be on a separate chip. A microphone 126, monitor 130, additional 
memory 134, and input devices (such as a keyboard and mouse 138), a network 
connection 142, and speaker(s) 144 may interface with input/output and control block 
118. Memory 134 represents a variety of memory such as a hard drive and CD ROM or 
DVD discs. It is emphasized that the system of FIG. 1 is merely exemplary and the 
irivention is not limited to use with such a computer system. Computer system 100 and 
other computer systems used to carry out the invention may be in a variety of forms, 
such as desktop, mainframe, and portable computers. 

For example, FIG. 6 illustrates a handheld device 160, with a display 162, which 
may incorporate some or all the features of FIG. 5. The hand held device may at times 
interface with another computer system, such as that of FIG. 5. The shapes and relative 
sizes of the objects in FIG. 5 and 6 are not intended to suggest actual shapes and 
relative sizes. 



Various memories mentioned above (e.g., CD-ROM, flash memory, hard-drive) 
include computer readable storage mediums on which in instructions may be stored 
which when executed cause some embodiments of the invention to occur. 

If this disclosure states a component, feature, structure, or characteristic "may", 
"might", or "could" be included, that particular component, feature, structure, or 
characteristic is not required to be included. If the specification or claim refers to "a" or 
"an" element, that does not mean there is only one of the element. If the specification 
or claims refer to "an additional" element, that does not preclude there being more than 
one of the additional element. 

Those skilled in the art having the benefit of this disclosure will appreciate that 
many other variations from the foregoing description and drawings may be made within 
the scope of the present invention. Accordingly, it is the following claims including 
any amendments thereto that define the scope of the invention. 



CLAIMS 

What is claimed is: 

1 . A method comprising: 

(a) calculating estimated weights for identified errors in recognition of 
utterances; 

(b) marking sections of the utterances as being misrecognized and associating 
the corresponding estimated weights with these sections of the utterances; and 

(c) using the weighted sections of the utterances to convert a speaker 
independent model to a speaker dependent model. 

2. The method of claim 1, wherein parts (a) - (c) are repeated at least once. 

3. The method of claim 1, wherein the utterances are converted into a 
recognized phone string a first time through applying the speaker independent model 
and thereafter through applying the most recently obtained speaker dependent model. 

4. The method of claim 1, wherein the estimated weights are computed 
through computing an average likelihood difference per frame and then computing a 
weight value by averaging the average likelihood difference over all the error words. 

5. The method of claim 1, wherein average likelihood difference per frame 
is used to calculate the estimated weights and is computed according to the equation (1) 
as follows: 

^ = -7r^S 1^ (1). 

Hg -Hf^ Re -Rf, 

where is the log likelihood of hypothesis word n, fi^ is the beginning frame 
index (in time), and is the end frame index, and /?£ , and R" are counter parts 
for a reference string. 

6. The method of claim 5, wherein the weight for misrecognized words of a 
particular speaker "i" is calculated according to equation (2) as follows: 

= - * f\Ln\ ' (2), 

wherein m a number of misrecognized words. 
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7. The method of claim 1, wherein for a particular speaker, different 
misrecognized words may have a different weight. 

8. A method comprising: 

(a) recognizing utterances through converting the utterances into a recognized 
phone string; 

(b) comparing the recognized phone string with a reference phone string; 

(c) calculating estimated weights for sections of the utterances; 

(d) marking errors in the utterances and providing corresponding estimated 
weights to form adaptation enrollment data; and 

(e) using the adaptation enrollment data to convert a speaker independent model 
to a speaker dependent model. 

9. The method of claim 8, wherein the utterances are converted into the 
recognized phone string through applying the speaker independent model. 

10. The method of claim 8, wherein parts (b) - (e) are repeated until 
differences between the reference and recognized strings are less than a threshold. 

1 1. The method of claim 8, wherein the utterances are converted into a 
recognized phone string a first time through applying the speaker independent model 
and thereafter through applying the most recently obtained speaker dependent model. 

12. The method of claim 8, wherein the estimated weights are computed 
through computing an average likelihood difference per frame and then computing a 
weight value by averaging the average likelihood difference over all the error words. 

13. The method of claim 8, wherein an average likelihood difference per 
frame is used to calculate the estimated weights and is calculated according to the 
equation (1) as follows: 

Ln—^„ ^ (1). 

where is the log likelihood of hypothesis word n, is the beginning frame 
index (in time), and is the end frame index, and Rl , ^b and Re are counter parts 
for the reference string. 
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14. The method of claim 13, wherein the weight for misrecognized words of 
a particular speaker "i" is calculated according to equation (2) as follows: 

^ n = l (2), 
wherein m a number of misrecognized words. 

15. The method of claim 8, wherein for a particular speaker, different 
misrecognized words may have a different weight. 

16. A memory comprising: 

a storage medium having instructions thereon which when executed cause a 
computer system to perform the following method: 

(a) calculating estimated weights for identified errors in recognition of 
utterances; 

(b) marking sections of the utterances as being misrecognized and associating • 
the corresponding estimated weights with these sections of the utterances; and 

(c) using the weighted sections of the utterances to convert a speaker 
independent model to a speaker dependent model. 

17. The method of claim 16, wherein parts (a) - (c) are repeated at least 

once. 

18. The method of claim 16, wherein the utterances are converted into a 
recognized phone string a first time through applying the speaker independent model 
and thereafter through applying the most recently obtained speaker dependent model. 

19. The method of claim 16, wherein the estimated weights are computed 
through computing an average likelihood difference per frame and then computing a 
weight value by averaging the average likelihood difference over all the error words. 

20. The method of claim 16, wherein average likelihood difference per 
frame is used to calculate the estimated weights and is computed according to the 
equation (1) as follows: 

Ln=-M ^ 

H^-H" R"-R" (1), 



12 



where is the log likelihood of hypothesis word n, H'^ is the beginning frame 

index (in time), and//" is the end frame index, and , and/?" are counter parts 
for a reference string. 

21 . The method of claim 20, wherein the weight for nnisrecognized words of 
a particular speaker "i" is calculated according to equation (2) as follows: 

wherein m a number of misrecognized words. 

22. The method of claim 16, wherein for a particular speaker, different 
misrecognized words may have a different weight. 

23. A memory comprising: 

a storage medium having instructions thereon which when executed cause a 
computer system to perform the following method: 

(a) recognizing utterances through converting the utterances into a recognized 
phone string; 

(b) comparing the recognized phone string with a reference phone string; 

(c) calculating estimated weights for sections of the utterances; 

(d) marking errors in the utterances and providing corresponding estimated 
weights to form adaptation enrollment data; and 

(e) using the adaptation enrollment data to convert a speaker independent model 
to a speaker dependent model. 

24. The method of claim 23, wherein the utterances are converted into the 
recognized phone string through applying the speaker independent model. 

25. The method of claim 23, wherein parts (b) - (e) are repeated until 
differences between the reference and recognized strings are less than a threshold. 

26. The method of claim 23, wherein the utterances are converted into a 
recognized phone string a first time through applying the speaker independent model 
and thereafter through applying the most recently obtained speaker dependent model. 
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27. The method of claim 23, wherein the estimated weights are computed 
through computing an average likelihood difference per frame and then computing a 
weight value by averaging the average likelihood difference over all the error words. 

28. The method of claim 23, wherein an average likelihood difference per 
frame is used to calculate the estimated weights and is calculated according to the 
equation (1) as follows: 

fie -fib ^e-f^b 

where is the log likelihood of hypothesis word n, is the beginning frame 

index (in time), and H" is the end frame index, and /?£ , and are counter parts 
for the reference string. 

29. The method of claim 28, wherein the weight for misrecognized words of " 
a particular speaker "i" is calculated according to equation (2) as follows: 

wherein m a number of misrecognized words. 

30. The method of claim 23, wherein for a particular speaker, different 
misrecognized words may have a different weight. 
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Abstract of the Disclosure 
In some embodiments, the invention includes calculating estimated weights for 
identified errors in recognition of utterances. Sections of the utterances are marked as 
being misrecognized and the corresponding estimated weights are associated with these 
sections of the utterances. The weighted sections of the utterances are used to convert a 
speaker independent model to a speaker dependent model. 
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DFCLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION 
(FOR INTEL CORPORATION PATENT APPLICATIONS) 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below, next to my name. 

I believe I am the original, first, and sole inventor (if only one name is listed below) or an original, 
first, and joint inventor (if plural names are listed below) of the subject matter which is claimed and 
for which a patent is sought on the invention entitled 

SPEAKER ADAPTATION USING WEIGHTED FEEDBACK 



the specification of which 



is attached hereto. - RFnEIVED 

was filed on (MM/DDA'YYY) 06/15/2000 as ni-vyi-i v i- 

United States Application Number „,., - . « onno 

or PCT International Application M.imhfir PCT/CNOQ/001 58 JUN i V L\i\iL 
and was amended on (MM/DD/YYYY) 



(if applicable) Technology Center 2600 

I hereby state that I have reviewed and understand the contents of the above-identified 
specification, including the claim(s), as amended by any amendment referred to above. I do not 
know and do not believe that the claimed invention was ever known or used in the United States of 
America before my invention thereof, or patented or described in any printed publication in any 
country before my invention thereof or more than one year prior to this application, that the same 
was not in public use or on sale in the United States of America more than one year prior to this 
application, and that the invention has not been patented or made the subject of an inventor's 
certificate issued before the date of this application in any country foreign to the United States of 
America on an application filed by me or my legal representatives or assigns more than twelve 
months (for a utility patent application) or six months (for a design patent application) prior to this 
application. 

I acknowledge the duty to disclose all information known to me to be material to patentability as 
defined in Title 37, Code of Federal Regulations, Section 1 .56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 1 19(a)-(d), of any 
foreign application (s) for patent or inventor's certificate listed below and have also identified below 
any foreign application for patent or inventor's certificate having a filing date before that of the 
application on which priority is claimed: 
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1 liereby claim the benefit under Title 35, United States Code, Section 1 19(e) of any United S 
provisional application(s) listed below: 


>tates 


Application Number 


(Filing Date - 


MM/DDr/YYY) 






Application Number 


(Filing Date - 


MM/DDA'YYY) 







I hereby claim the benefit under Title 35, United States Code, Section 1 20 of any United States 
application (s) listed below and, insofar as the subject matter of each of the claims of this application 
is not disclosed in the prior United States application in the manner provided by the first paragraph 
of Title 35, United States Code, Section 1 12, 1 acknowledge the duty to disclose all information 
known to me to be material to patentability as defined in Title 37, Code of Federal Regulations, 
Section 1 .56 which became available between the filing date of the prior application and the national 
or PCT international filing date of this application: 



Application Number (Filing Date - MM/DDA^YY) Status -- patented, 

pending, abandoned 



Application Number (Filing Date - MM/DDA'YYY) Status - patented, 

pending, abandoned 
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I hereby appoint the persons listed on Appendix A hereto (which is incorporated by reference and a 
part of this document) as my respective patent. attorneys and patent agents, with full power of 
substitution and revocation, to prosecute this application and to transact all business in the Patent 
and Trademark Office connected herewith. 

Send correspondence to John Patrick Ward , BLAKELY, SOKOLOFF, TAYLOR & 

(Name of Attorney or Agent) 
ZAFMAN LLP, 12400 Wilshire Boulevard 7th Floor, Los Angeles, California 90025 and direct 

telephone calls to .lohn Patrick Ward , (408) 720-8300. 

(Name of Attorney or Agent) 

I hereby declare that all statements niade herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made 
are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United 
States Code and that such willful false statements may jeopardize the validity of the 
application or any patent Issued thereon. 

Full Name of Sole/First invpntnr YonahonqYan _ 



lnw^ntnr-.^<^lgnature ^^^^^S^A ^ Date /o? . ^ 

Residence Beaverton. Oregon Citizenship P. R. China ^ — ^ N r 

(City, State) (Country) 



Post Office Address 20756 NW Amberview Lane 



Beaverton. Oreoon 97006 
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APPENDIX A 



William E. Alford, Reg. No..3ZrZ64; Farz'ad E. Amini, Reg. No.' 42.261; Peggy S. Avalos, Reg. No. 42^4; 

William Thomas Babbitt. Reg. No..3a.^1; Carol F. Barry, Reg. No. 4LfiQ0; Jordan Michael Becker, Reg. 

No 39 602; Lisa N. Benado, Reg. No. 39-295; Bradley J. Bereznak, Reg. No. 33,4Z4; Michael A. 

Bernadicou, Reg. No. 35,934; Roger W. Blakely, Jr., Reg. Na25j831; R. Alan Burnett, Reg. No. 46,149; 

Gregory D. Caldwell, RegTNo. 39^926; Jae-Hee Choi, Reg No. 45^288; Thomas M. Coester, Reg. No. 

3g,637: Robert P. Cogan, Reg. No. 25,049; Donna Jo Coningsby, Reg. No. 41,gMuFlorin Corie, Reg. No. 

4^233^Mimi Diemmy Dao, Reg. No. 45,628; Dennis M. deGuzman, Reg. No. 4J^Z02; Stephen M. De 

Klerk, Reg. No. 4e,5Q3: Michael Anthony DeSanctis, Reg. No.^957; Daniel M. De Vos, Reg. No. 

37 813; Justin M. Dillon, Reg. No. 42A86; Sanjeet Dutta, Reg. No. 46,145;JVIatthew C. Fagan, Reg. No. 
Tarek N. Fahmi, Reg. No. Iljfl2; Thomas S. Ferrill, Reg. No. 42,532; Mark J. Fink, Reg. No. 

45.2Zfl; George Fountain, Reg. No. 37,374; Andre Gibbs, Reg. No. 47.,5aSLJames Y. Go, Reg. No. 

40,621; Alan Heimlich, Reg. No. P4BSSB; James A. Henry, Reg. No. 41,064; Libby H. Ho, Reg. No. 

4i!^5 Willmore F. Holbrow III, Reg. No. 41 ,845j ^heryl Sue Holloway, Reg. No. 37,850; George W 

Hoover II, Reg. No. :^992; Eric S. Hyman, Reg. No. 30,1 39; William W. Kidd, Reg. RoTSLZZS; Sang Hui 

Kim, Reg. No. 40,450; Walter T. Kimv Reg. No. 42.7-3lT Eric T. King, Reg. No.j44J88; Steve Laut, Reg. 

No. 47,736; Georp"Brian Leavell, Reg. No. 45,436j Samual S. Lee, Reg. No._42791 ; Gordon R. Lindeen 

111 Reg. No. 33,192; Jan Carol Little, Reg. No..4Um; Julio Loza, Reg. NO;^7,7^Joseph Lutz, Reg. No. 

43,765; Michael J. Mallie, Reg. No. 36,591 ; Andre L. Marais, Reg. No. 48,095; Paul A. Mendonsa, Reg. 

Nal2ra7-9; Clive D. Menezes, Reg. No. 45,493; Richard A. NakashimaTRegTRo. 42,023; Stephen Neal 

Reg. No. 41,315; Chun M. Ng, Reg. No. 3gig78; Thien T. Nguyen, Reg. No. 43,835; Tf iinFi V. Nguyen, 

Reg. No. 42,034; Robert B. O'Rourke, Reg. No. 46.972: D aniel E. Ovanezian, Reg. iTo. 41 ,236; Kenneth 

B Paley, Reg. No. 38,989; Gregg A. Peacock, Reg. No._45^1; Marina Portnova, Reg. No. 45,750;_ 

Michael A. Proksch, Reg. No. 43,021 ; Randol W. Read, Reg. No. 43,876; William F. Ryann, Reg. 44,313; 

James H. Salter, Reg. No. 35£e8; William W. Schaal, Reg. No. 3^^ames C. Scheller, Reg. No. 

31 195; Jeffrey S. Schubert, Reg. No. 43,098; George Simion, Reg. NorP47,089; Maria McCormack 
S^i^rino, Reg. No. 31 ,639 ; Stanley W. Sokoloff, Reg. No. 25,128 ; Judith A. Szepesi, Reg. No. 39,393; 
Ronald S. Tamura, Reg. No. 43j179; Edwin H. Taylor, RegrRor25,129; Lance A. Termes, Reg. No. 

43.184: John F. Travis, Reg. No. 43,2QaLi<erry P. Tweet, Reg. NoT^^^LMark C. Van Ness, Reg. No. 
39 a65 °Tom Van Zandt, Reg. No. 43.219; Lester J. Vincent, Reg. No. 31.460: Archana B. Vittal, Reg. No. 
3ijja2;,Glenn E. Von Tersch, Reg. No. 41 .364: John Patrick Ward. Reg. No. 40,216; Mark L. Watson, 
Reg. No. 46,322; Thomas C. Webster, Reg. No. 46,154; and Norman Zafmari, Reg.' No. 26^0; my 
patent attorneys, and Firasat AM, Reg. No. 45.7r5r Ch'arles P. Landrum. Reg. No. 46,855;_Suk S. Lee, 
Reg No. 47,745: and Raul Martinez, Reg. fio,A&mA, Brent E. Vecchia, Reg. No. .P48,01 1 ; Lehua Wang. 
Reg. No. P48.02 3; my patent agents, of BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN LLP, with offices 
located at 12400 Wilshire Boulevard, 7th Floor, Los Angeles, California 90025. telephone (310) 207-3800, 
and Alan K. Aldous, Reg. No. 31.905; Ben Burge, Reg. No. 42,372 ; Robert A. Burtzlaff, Reg. No. 35,466; 
Richard C. Caldenwood, Reg. No. 35,468; Jeffrey S. Draeger, Reg. No.^li^O; Cynthia Thomas Faatz, 
Reg No . 39,973 ; Jeffrey B. Huter, RegTNo. 41 ,086; John Kacvinsky. Reg. No^JO^O; Seth Z. Kalson, 
Reg NoT4a670; David J. Kaplan, Reg. No. 4l?r05r Peter Lam. Reg. No^,fi55; Charles A. Mirho, Reg. 
No 41 1 99- Paul Nagy, Reg. No. 37,896; Leo V. Novakoski, Reg. No . 37.1 9J ; Thomas C. Reynolds, Reg. 
No 32^8; Kenneth M. Seddon.'RigrNo. 43,105; Mark Seeley, Reg. No.,32. 299 ;.Steven P. Skabrat, 
Reg~Tio'36JZ9; Howard A. Skaist, Reg. Nor3g:008; Robert G. Winkle, Reg. No . 37,474;. Sharon Wong, 
Reg. No!^!^; Steven D. Yates, Reg. No. ^2^?a'vin E. Wells; Reg. No. 43,256and Charles K. 
Young, Reg. No. 3a.4.354-my patent attorneys, and my patent agents, of IN TEL CO RPORATION, with 
offices l ocated at 2200 Mission College Blvd.. Santa Clara, CA 95052^telephone (408)765-8080; with^ full 
poweTofsubstltution and revocation, to prosecute this application and to transact all business in the 
Patent and Trademark Office connected herewith. 
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APPENDIX B 

Title 37, Code of Federal Regulations, Section 1 .56 
Duty to Disclose Information Material to Patentability 

(a) A patent by its very nature is affected with a public interest. The public interest is best served 
and the most effectiye patent examination occurs when, at the time an application is being examined the ' 
Office is aware of and evaluates the teachings of all information material to patentability. Each indiyidual 
associated with the filing and prosecution of a patent application has a duty of candor and good faith in 
dealing with the Office, which Includes a duty to disclose to the Office all information known to that individual 
to be matenal to patentability as defined in this section. The duty to disclose information exists with respect to 
each pending claim until the claim is cancelled or withdrawn from consideration, or the application becomes 
abandoned. Information material to the patentability of a claim that is cancelled or withdrawn from 
consideration need not be submitted if the infomiation is not material to the patentability of any claim 
remaining under consideration in the application. There is no duty to submit information which is not material 
to the patentability of any existing claim. The duty to disclose all information known to be material to 
patentability is deemed to be satisfied if all information known to be material to patentability of any claim 
issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§1 97(b)-(d) 
and 1.98. However, no patent will be granted on an application in connection with which fraud on the Office 
was practiced or attempted or the duty of disclosure was violated through bad faith or intentional misconduct 
The Office encourages applicants to carefully examine: 

(1 ) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest infomiation over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to Information 
already of record or being made of record in the application, and 

(1 ) It establishes, by itself or in combination with other information, a prima facie case of 
unpatentability of a claim; or 

(2) It refutes, or is inconsistent with, a position the applicant takes in: 

(1) Opposing an argument of unpatentability relied on by the Office, or 

(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when the infomiation compels a 
conclusion that a claim is unpatentable under the preponderance of evidence, burden- 
of-proof standard, giving each term in the claim its broadest reasonable construction 
consistent with the specification, and before any consideration is given to evidence 
which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the meaning of 
this section are: 

(1) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by disclosina 
infomiation to the attorney, agent, or inventor. 

(e) In any continuation-in-part application, the duty under this section includes the duty to disclose 
to the Office all information known to the person to be material to patentability, as defined In paragraph (b) of 
this section, which became available between the filing date of the prior application and the national or PCT 
international filing date of the continuation-in-part application. 
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